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ABSTRACT 

In 1983, in the Netherlands* highly differentiated 
school system, two types of curriculum representing different ability 
levels were combined as a first step towards a more heterogeneous 
grouping of student abilities. A study of one aspect of the results 
of this change compared over 1000 samples of English and German 
second language listening comprehension tests administered before and 
after the change. Statistical analysis of test results from students 
representing two different ability levels in different years indicate 
that the curriculum change may have had a negative effect at both 
levels of achievement. The results indicate that mixing ability 
levels has a negative effect on the final ability attained. The 
findings are supported by results on tests of other foreign language 
skills (reading comprehension) and in another subject (chemistry). 
(MSE) 
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EFFECT Of HIXIHG ABILITY GROUPS (W ABILITY LEVELS ATTAImtD 
JOKN H.A.L. da Jong (C1to, Ketherlands J 



ry 

In the Ketherlands the system of secondary *#*at1en1« highly 
differentiated. Possibly ts a first step towards a cemmcebemlve 
system two different types of secondary educative were fcrwpit 
together In 1983. However, because of the width of the ability reoge. 
examinations at two different levels are offe'/od at the oiilof .the 
curriculum. Tht two ltvtls were eeant to bt equivalent to these in 
the two types of secondary education of the former system. Data from 
sanies of 300 to 1000 examinees on English and Careen forel^ 
language listening comprehension tests ware collected before emd 
after the introdictlon of the - v systea. In en eeuatlng eroceoVire 
based on Item Response Theory, using the Rasch Model , tests of 
different levels and from different years are Interlinked by means of 
common Item and common representative sables. The chanee in the 
educational systeai Is found to have had a negative effect at both 
levels of achievement. These findings are corroborated by results on 
tests of other foreign language skills (reading comprehension) and 
other subjects (chemistry). 



Introduction 

Secondary education In the Motherland* is a highly 
differentiated systeai. Four different school types ale at four 
different proficiency levels. 1A-iJ . . 

LB0 provides lower vocational training and Is attended by about 
ZZX oTThe age cohort. LBO has a four year programme* Final 
examinations are taken as a rule at the age of 16. T7* foreign 
language programme takes about 300 hours and leads to a level 
comparable to the Threshold Level as defined by the Council of Europe 

(Va " HAV0 l 1s 5 the lowest level of gtnoral education, meant as a i basH 
for flHner training for clerical Jobs and Is attended by about 37% 
of the age cohort, IttYO has a four year programme. Final examinations 
are taken at the age of 16. The foreign language programme takes 
about 400 hours. m , . _ . . 

HAVO 1s an Intermediate level of general education, meant to 
preparFTor further studies leading to hlfrer non-academic Jobs and 
Is attended by about 22t of the age cohort. Final examinations are at 
the age of 17. The foreign languege program** takes about 500 hours 
and the final level 1s comparable to that reached by secondary 
education 1n nost western countries. 
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VWO is the highest level of general education want as a 

iuS . academ,c stud,es *nd is taken by about 181 of the age 
52? f.I S J S,x year P r °9r«we. Final examinations are taken at 

a 2? th ° n ?h. !;.^n f0^e 1 I 9n , , 4 091,496 P r °9~ takes about 600 hours. 
a, », ? HAVO schools, however, two subtypes have to be 

distinguished. Until 1979 pupils could choose either a three or a 
four year program: HAVO-S and HAVO-4. Fin" examination^ MAVO-3 
were at a level below the regular (i.e.: HAVO-4) level. This HAVO-3 
level was comparable to the LBO level and was taken by about » of 
all pupils attending HAVO. For several school subjects examination 
papers were Identical for LbJ and rtAVO-3. In 1979 It was decided to 
bring al 1 HAVO pupils together In a single school w th h * Tfour 
year currlculu* for all. At the end of the currlculu-n pupl .s have the 
cnolce - for each subject separately - to take examinations at No 
different levels: HAVO-C or HAVO-9. These levels are meant to 
correspond to the HAVO-o and HAVO-4 levels m the former system and 
IVffr "anlnatlon papers are once again ^dontlcal for LOO and 

of f»r in /I » nC : e ^ h,nd the Chan9e of tPe VS tern for HAVO Is to 
offer all students better chances. A HAVO-3 certificate was not 
greatly ipprectated. It was therefore felt to be better for HAVO 
students to be able to postpone the definitive choice for a 
P m , IW' f° °f fer a " MW stude,!ts the opportunity to 

h^ln mL" 9h6r ,eve ' MAV0 certificate. If this aim should prove to 

I 9 lY SOme> then the Possibility of taking exams In a 
number of subjects at the lower level would give them j chance to 
obtain a certificate with the prestige 0 the* h gheT,evel HAVO for 
at least the remainder of the subjects. 

In 1983 the first cohort of HAVO pupils In the new system 
presented thenselves for final examinations. The present study was 
undertaxen to revea the effect of the changes in the HAVO schools 

?2»if e ^ Me lf e,n !? t ,ev ! 1 0f H * V0 pup,,s: d,d the postponement of 
evel determination and the lengthening of the curriculum for the 
lower level lead to better results and to higher certification? 



•let hod 



D..,i. ^ e J m f th0d used ,lL based on Item Response Theory using the 
Rasch Model (Rasch, 1960). m the Rssch Hodel one Item parameter and 
one person parameter determine the probability that a given person 
answers a given Item correctly. The item parameter In the Rasch 
cluL thp 4 lin,i he d,ff, "'ty P^ter and the person parameter Is 
called the ability parameter. However, since for the calibration of 
each test an arbitrary origin of meas—ement is selected the 
nunerlcal values of the parameters of different tests cannot be 
h^ 6 !^! 011 ^ '\ 0rder t0 make them comparable a common origin 
has to be determined for the different tests. This procedure known 
as test equating, can be followed if: »<-eeure, xnown 

- the tests contain 3 number of common (tens; 

- the groups taklrr) the t»sts cnntiin common 'persons ; 
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- the jro*s taking tht ttsts art rtpretentttlvt tanplts frai the 

If*£ WiS" condition, 1* «t. and «f tht U.ts -»ur. th. s~ 
ability, and tht tt$t$ are not to far apart In difficulty * M 
rtllablilV (Ptttrstn tt al.. 1982) tht 1tt« pjrtwttrs o? tht ttsts 
and tht ability paraaettrs of tht ptrsons can bt tquattd. I.t. tnt 
paretettrs can bt put on a single 

In 1982 Una Hit ytarof WVO txa^nt^ons 1n^ forttr 
tvttM) rind cm stasias of 300 wart taktn frot tht 110, PWru-3 ano 

pSCl"lon7 Btctust tht L80 and r*W-3 Ttvtls ^J"* to 
be Idtntlcal 160 and H/WO-:. students tokt tht MM fort 
llsttnlng coaprehtnslon ttst. Tht students In tach Mjli^t asktd 
to tokt one third of tht ttst of tht othtr level btsldts Utlng tha 
est at Zil -own lml: thus LBO and rtA.W-3 studtnto to* <r pert of 
theHAVO-4 ttst and KAYO -4 students took t ptrt of tht 
ttst. In 1983 and 1984 slartlar proctdort* wart followtd with saaiplts 
of 300 to 1000 frm tht LBO. MWO-C and MAVO-0 populations. This 
design would pervit equating both tht tests and tht W" 1 ™ 0 "* 
cVncerntd by Sans of t cc-on Ito. 1 nk. Rando. stjjplts of 1000 L80 
pupils fro. the yters 1982. 198? and 1984 were ttktn to for. 
represenutlvt sables of tht saa» population, thus permitting to 
equate the scales of the substqutnt years. 

Tht tests used In this study art tht Cltotttts of 

forelon language listening comprehension. The tests were originally 
develSed in a . ^starch project at the University of Utrtcht IG.oot. 
1975). Beceost txt-lnatlons art public. ntw ttsts •»<"}*■• 
developed etch year. Clto ttsts of foreign language 1 
cotprthenslon have been constructed at Clto and used In flnil 
examinations In schools for over toy years. !»>• «*Jtct ve of tot 
tests as dtflntd by Groot (1975), Is to tvaluatt foreign language 
learners' ability to understand tht fortlgn langoaot, aP*™ . . 
SDontantously by educated native speakers at nonaal convtrsttlonjl 
ISed '"reiely Infomal elements ai -til ts ltxlcal and syntactical 
efeSnts t£Tart Incomprehensible to less 

and topics requiring specific knowledge are excluded from the tests. 
The ffiuage •terlal consists of three or four InUrvlews with 
nat1vtTpe*ers of various occupations and profess c-ns. Tht lt« 
fonaat Is t wltlple cholct qutstlon with thrtt options. Tht language 
iaterlal Is divided In sables of forty to fifty "foods; 
answer s a one phrase sue*ery of the global contents o f .^ e t « 1e - 
E»ch smoIc Is followed by a pause of twenty seconds or, tht tope 
Sen allows the stmts' to Sad the ltd. In thtlr tost book end 
tick their answer on an answer sheet. A second Itot format was 

troduteS (Se Jong. 1983; Oe Jong. 1984): JV^'h?^. 
tw SpUons! In each sample of twenty to thlr* "J*" 1 * 8 

word or oroup of words Is deleted and replaced by an electronic 
signal? V^to bTdeleted fron the original text am chosen for 
their sewntlc relevance In the context. The language •twl« 
onslsts of discussions, conversations news 1te«s and other radio 
oroorsws. The students are to decide In a pause of seven seconds 
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on 1ml art 1t« fori? tot^l Situ 2? <)H91n,1 text - ">«Pendlng 
IS ""*™5 twS) ranges % ee 7 n 5 f CM"' ^ 



*etn1t* Md dltomlon 



tnt Jtarj »«, 1983 Md 1984 The Vhr^^ ePS i 0 , n tests for NAVO In 
»■ represent the ability variable f horizontal lines In figure 

calibrate* on . jImTTsc^* Abowlh. i?' ,S ^ he tests have been 
higher 1m i ,? indicted Soriano lei J Tk? r2L 4b11 ^ at «• 
represent the distribution of JJJifH! 2 s s- Th * shaded areas 
belo. tht wen to tao^UndSrd^I^i fr0 " 5"° standard options 
covering about 96W £ «on„i^ ,t1ons ,bove the thus 

Tht dot on «*! 11ne7ep7ese£ ! ne SuVf ^ WV0 leve1 - 
aubseouent vear* mm 2*. . . 0 " ean «pWty In the 

unity of wesurMent on the iblllS IrlSi ^Li 01 " 0ne lo 9 U lth « 
levels. Thl* difference rm»ul f ' 5 ctween the HAVO 
fan. The wan .buij JHoS Sn TeEu^*"!? the ttree 
1»: to the lowr end of teTm, J e !' sh '£ s to the left. that 
ability .t both laveH sl%*J««s J J: J?"; though the mean 
doe, not reach the original lmPoMge ' '* in ' n 1984 U 

vartes fr3^. r r°to^,*r nt ; ?! *1 ^j? 1e « 1s »• «« equal and 
population hH t.Sn^^ o^^^ «S e , tOt ? , ? AV0 
on. This has an effect on the wan aMiM^S 1P W level frm 1983 
population. The nan abllltv and tJ? luJ r < S.\, t,ie total ^ 
total MVO population T B nZXV abi ,"* U the 
the saw scale as rigure li. TheVr,** 9 ^}.; F ^ ure lb 1s drawn a t 
the subsequent years? the «ha*S indicate the wan ability In 

ability of 96» of (he total iSftS ^u^^L^ ««rtbutlin of 
the shift to the lowr^nd of 2. P 2« , i! 1on <both leve ' s >- ADart 
ability can be observed *" tty scale a spread < n 

figures 2a and 2b picture th» <». w ^ * j ^ 
"stoning ceoprehensfon. Figure 2a sh£s "c^m f ° r * n9 " sh 
W end of the ability scilYas JrSLL c £ parab,e sh1ft to tt « 

a sMght rtirto^JN^^n^^ln. as for 
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c»Pl«tt recovery of tht original lml Tht eo*1*tt1o* 6f tht two 

"J!?,! 6 *!!* F<9ur * 25 $h "" 1 Jllrtlar Mrmlnf of tut spread m 
ability of tht total HAVO population as not foond for Otnttn. 

An observed shift In e*vn ability of a population of atatt 
90,000 students cannot bo Interpreted as • standard orrof of tut 
toan. Furtherwrt, 1t Is MJhU Wobobie that such a staottN error 
of tht wan would causo devTatltnT; tht saw direction fjrtwo 
different years and for two different langoaott. To Choc* on 10*11* 
errors rtference was .ade to the rtsults TTe populations concerned 
on reading comprehension tests. Atsults *n tht rtodlnf cotfeohontlon 
tests art collected each year froo thw total population!^ 000 {Jo 
students and 90,000 havo students. Flfwre 3 pictures the rolatlve 

ffii^tSL 11 ?* yvmr * l!* 1 * vo »todtnu and tht UK) students froa 
1978 to 19* In standard deviations of tttt ttoros. Tht wean UN) 
score Is set at xero, tht horizontal axis represents tint a*d the 
vertical a^fs sta.xlard deviations of test scores on tht rooting 
coe^rebenslon tosts. Tht etan score of the lower It vol HAW ftodents 
for English reading cooprehenslon was traditionally well e*0vo tht 

* or * of "° * tw * nts - ™» situation changes in 19*3 1 tht lower 
level HAV0 students drop to a level that Is sort then half a standard 
deviation below their fortor 1ml, For fltnaan e slaiiir eVop can t» 
observed, In this cast enlarging tht original distinct bttottA IBO 
and lower level HAVO. 

Another possible Influence on the totn aDlllty Of tht HAVO and 
180 students co Id be a change In the groups invdlm. If trite a 
Urge proportion of the age cohort has choten for t dlfftftnt 
schooltypt since tht change of the systea, a shift In oean ability of 
the croups Is to be expected. Figure 4 picture* tht meter tf 
exanrfnees taking English (ca. 97l of the total population) In the 

?iE e I™h£* oTtypts frm i972 * lfM - lootlngit tht toHOd fro* 
1972 to 1932 a constant grewth, parallel to the postwar Increase In 
population can be observed* Tht two HAW groups are an exception m 
that the lower level MAYO group decreases frow 1900 on going with a 
■ore rapid growth of the higher level HAVO. lamdlaUly after the 
change of the HAVO syste* a drowetlc change In the two IKVO 
populations can be observed: aore than threo tloes as tony HAVO 
students opt for the lower level HAVO* Thert 1s no Indication of a 
shift In population between LBO and HAVO. The larger nutter of lower 
level HAVO students Is apparently recruited fre* the roup that 
formerly opted for the higher level KAYO and would thtrefort be 
expected to raise the level of achievement In the lower level group. 
Furthermore, hypothesise that It will not have been the highest 
achiever* In the higher level group that deserted this level for the 
lower, one could equally have expected a rise In K an ability for the 
Mojer level HAVO. The observed change 1n nutters of the HAVO groups 
therefore, cannot account for the downward shift In achievement. Cn 
the contrary, this change would lead one to expect a shift In the 
opposite direction. Bakker (1984) reports that the hypothesis of a 
rise In ewan ability of the higher MAYO group In chemistry, due to 
the changed distribution of MAYO students over the two HAVO levels 
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•fter tht change of the system, has to bt rejected. 

Apart from the tf ftct on the achievement level of HWO students 
the cfctwo 1» tlit MAYO system has had Us effect on their level of 
certification. Flfure $ picture* this effect. Tht proportion of MAYO 
•Jr 1 " 1 ? t* 1 "* * «*J»ct at tht higher KAYO level has gone down for 
ill ££te£ , V H *!?r? « lct proportions In 1982 and 1983 

for Emjllt*. Far tht fortign languages the proportion of C a. .05 of 

v ^ l t^Ly?^_ p0 y i1 1 ? W t ?S t ?* % 1<wer "* w level has gone up 
toa proptrtlt* tf .15 to .18. For subjects such as mathematics and 
chemistry, traditionally regarded as difficult, the original .05 my 
fo wp to proportions of .20 to .40. ^ 

Tht ad voce Us of tht change In the MAYO systen present the 
statistics tf Figure 5b is proof of a positive effect. Before the 
change of the yttm 15% to 201 of tht total KAYO population finished 
their lest year without any certificate at all and had to sit the 
Ustyttr again to obtain a certificate, which meant the loss of an 
22 t l f ?.* Mr for **** * n the new systen, only about 7% or 

« fall to obtain I certificate. However, one whole year has been 
ttted to the curriculum for about 5% of tht students: they h«ve had 
their "lets of a year". 

^ To am*t • f«ir comparison between the certification of MAYO 
,t !^^!£ , ai * tfttr **• of tht system, the level of 

certification also has to bt taken Into account. Figure 5c shows that 
the proportion of Have students taking all subjtcts at tht lower 
Itvtl has remained more or less the same 1n 1982 end 1983. The number 
SL f ! n fT2 <BW f ,t ^li youp hes Increased dramatically: from about 
ZQH In 1982 to almost 501 In 1983. The proportion of candidates 
Uklng MAYO exams 1n all subjects at tht higher level In 1983 Is only 
htlf of that tf tht years before. Granted, the students taking all 
subjects at the hlghtr level do so with success In 1983 a larger 
proportion pastes tht exams, but It Is much smaller group than 
htfort. A smaller proportion of MAYO students will In eff>ct enjw 
the greater social appreciation associate* with the higher level 
certificate. The proportion of kayo students passing at the higher 
level MAYO for all subjects has gone down from about .80 In 1982 to 
about .53. letwten lower level MAYO and higher lev-1 MAYO 
Intermediate levels havt bten erected; students can n<w have mixed 
level ctrtlficttts. Thest mixed certificates can contain one subject 
at the lower level and five subjects at the higher level; or five 
subjects at tht lowtr level and one subject at the higher level or 
any other combination with a total of six subjects. How exactly the 
mixed level certificates will be appreciated cannot be foreseen, but 
a certificate with all subjects taken at the higher level will no 
doubt receive more appreciation. 

After oflly two years It Is too early to pronounce a final 
Judgement on tht new MAYO system. However, the downward shift In the 
mean ability of the total MAYO population for listening comprehension 
of German «nd English, the relat1vel> lower scores for the lower 
level MAYO on reeding comprehension tests of German and English, the 
absence of a rise In wean ability In chemistry In spite of a more 
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select group, tht smaller proportion of SAVO students obtaining s 
certificate wltfc all subjects U*en at tut higher ltvtl, all timet . 
observations reveal t similar tendency: *4x1ng ability r»M h«s a 
negative tfftct on tht final ability attaint*. Various explanation* 
for this negative tfftct can be of ftrtd. Ttechars mm/ ttni to adapt 
thtlr level of teaching to tht wan lml of tht stedtats 1* tht 
group. Students my bt less motivated to d» thtlr utmost *fctn thmy 
kn« failure at tht hlghtr level dots tot mean total failure: th* 
can always try thtlr success at a loner Itvtl. Though Here dita on 
the results of tht nt* system art need e d, it Is Impossible to welcome 
the present rose Its and estreat Ctttton has t6 bd taken In further 
experiments towards a comprehensive system. 
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